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ABSTRACT* 

Objectives: The purpose of this study was to gather 
longitudinal data on a pharmacy class cohort 
concerning programmatic components as well as 
develop a profile of student experiences as they 
progress through the curriculum. 
Methods: The Class of 2006 (n = 67) completed a 2 
page instrument at the conclusion of the first 3 
professional years (PY) concerning hours spent in 
various activities, overall quality of various 
programmatic components, relationships with others 
in the college, and employment information. During 
senior week, a more extensive exit survey was 
administered. 

Results: At the conclusion of PY1 , 56.5% of the 
class was working as a pharmacy intern. By PY3 
this increased to 88.1% with a decrease to 65.7% in 
the final year. The hourly range of hours worked 
followed the same pattern. The rating of Dean's 
Office Staff and interactions with faculty members 
remained constant across all 4 years. In the final 
exit survey the 2 lowest rated program components 
were the quality of the interaction with assigned 
faculty advisor and the availability of professional 
electives. There was no difference across the 
professional years for the quality of relationships 
with staff or faculty; however, the mean rating of the 
quality of relationships with other students was 
higher for PY4 when compared to PY1 . 
Conclusions: College faculty, administrators, and 
committees have used the information gathered 
from students in program assessment and 
enhancement efforts. Longitudinal data collection 
allows for tracking of changes and interventions. 

Keywords: Longitudinal Studies. Students, 
Pharmacy. Education, Pharmacy. United States. 



RESUMEN 

Objetivos: El proposito de este estudio fue recoger 
datos longitudinales de una cohorte de una clase de 
farmacia relativa a los componentes programaticos 
asi como desarrollar un perfil de las experiencias de 
los estudiantes y su progreso durante el curriculo. 
Metodos: La clase de 2006 completo un 
instrumento de 2 paginas al acabar los 3 primeros 
anos profesionales (AP) relativo a las horas 
empleada sen varias actividades, calidad general de 
los diversos componentes programaticos, relacion 
con otros en la facultad, e informacion sobre 
empleo. Durante la semana de los mayores, se 
administro un cuestionario mas extenso. 
Resultados: Al final del API, el 56,5% de la clase 
estaba trabajando en una farmacia como interno. En 
el AP3 esto se incremento al 88,1%, con una 
disminucion al 65,7% en el ultimo afio. El rango de 
horas trabajadas siguio el mismo patron. El ratio de 
interacciones con la oficina del Decano y con el 
personal de la facultad permanecio constante 
durante los 4 anos. En la encuesta del final los dos 
componentes menos valorados fueron la calidad del 
a interaction con el consejero del a facultad 
asignado y la disponibilidad del as opciones 
profesionales. No hubo diferencia durante los anos 
profesionales en cuento a la calidad de la relacion 
con el personal o los profesores; sin embargo la 
media de puntuacion del a calidad del a relacion 
con otros estudiantes fue mayor en el AP4 que en el 
API. 

Conclusiones: Los docentes del a facultad, los 
administradores y los comites han utilizado la 
informacion recogida de los estudiantes para 
evaluar el programa y los esfuerzos. La recogida 
longitudinal de datos permite seguir los cambios y 
las intervenciones. 

Palabras clave: Estudios longitudinales. 
Estudiantes de farmacia. Education en farmacia. 
Estados Unidos. 
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INTRODUCTION 

Echoing Standards 2000, the new Accreditation 
Council for Pharmacy Education (ACPE) 
Accreditation Standards in Guideline 22.3 states 
that student perspectives should be obtained via 
various methods such as exit surveys and, 
"...should be systematically analyzed and used to 
improve all aspects of the program and allow for 



www.pharmacypractice.org 



109 



Draugalis JR, Plaza CM. Programmatic assessment of student experiences using a longitudinal survey. Pharmacy 
Practice 2007;5(3):109-1 14. 



longitudinal and cross-program evaluation". 1 ' 2 ACPE 
Standards No. 15 - Assessment and Evaluation of 
Student Learning and Curricular Effectiveness, 
Standard No. 21 - Program Information, and 
Standard No. 22 - Student Representation and 
Perspectives call for assessment of the student 
perspective on the harmony of relationships among 
students, faculty, and staff, availability of financial 
aid information, and the quality of the Student 
Affairs Office among other programmatic 
components. Also little is known about how students 
spend their time in the "typical" week or the types of 
external pharmacy work experience gained if any as 
they progress through the curriculum. The 
significance of this longitudinal analysis of student 
perspectives and experiences is important given 
categories and competency statements in 
Standards 2007 and the lack of information on how 
students spend their time and experiences and 
competencies achieved outside the classroom in 
venues such as health fairs and work experience as 
they progress through the curriculum. The literature 
contains examples of how schools and colleges of 
pharmacy comply with other components of the 
ACPE Accreditation Standards but not the elements 
examined in this study. Using a reasoned approach 
in assessing student experiences and growth can 
produce valid data to be used in "guiding policy and 
decision making, improving practice, and 
demonstrating effectiveness to external 
authorities". 3 The purpose of this study was to 
gather longitudinal data on a pharmacy class cohort 
concerning programmatic components as well as 
develop a profile of student experiences as they 
progress through the curriculum. 

METHODS 

The existing anonymous annual student exit survey 
developed by the then Associate Dean for 
Assessment and Evaluation at the University of 
Arizona College of Pharmacy (UACOP) was 
modified in spring 2003 to track students across the 
didactic portion of the curriculum. The instrument 
was administered to students in the graduating 
class of 2006 at the conclusion of each academic 
year beginning the spring semester of professional 
year 1 (PY1). Students present in class the day of 
the administration were asked to participate. The 
anonymous database contains information on four 
areas: 1 ) number of hours spent weekly in various 
activities (eg, studying, socializing, caring for family 
members, and working), 2) quality of various 
programmatic components (eg, availability of 
financial aid information, availability of Dean's office 
staff), 3) rating quality of relationships among 
students, staff, and faculty, and 4) whether students 
are currently employed as a pharmacy intern and in 
what type of practice setting (eg, community, 
institutional). The adapted exit survey contained 
core sections administered across all years as 
follows: 7 items related to the number of hours 
spent in various activities, 6 items concerning the 
quality of various programmatic components, 3 
items on the quality of relationships among 
students, staff, and faculty, and 3 items related to 
pharmacy experience. The survey administered 



during the first professional year had 3 additional 
items beyond the core sections on the quality of 
various programmatic components (eg, pre- 
pharmacy advising, admissions, and orientation 
week) unique to that year in the curriculum. The 
complete exit survey was administered to the class 
of 2006 which in addition to the core sections 
contained an additional 8 items addressing the 
quality of various programmatic components as well 
as 7 items concerning the administrative, staffing, 
and resources related to the experiential component 
and 5 items on the longitudinal drug information to 
address the portion of the curriculum unique to the 
final year in the program. As well open-ended 
questions asked graduates to provide responses on 
areas needing improvement in the longitudinal 
program. This information was used as a 
component of programmatic evaluation at the 
UACOP. Given the completely anonymous nature of 
the survey, each administration was considered a 
cohort that provided a cross-sectional snapshot. A 
longitudinal analysis was conducted of trends 
across different items at the cohort level since it is 
impossible who to know responded on an individual 
level. 

A one-way ANOVA was used to analyze continuous 
level data across multiple year cohorts with a Tukey 
or Dunnett post-hoc test for significant F-tests. An 
independent t-test was used for continuous level 
data where only 2 cohort years were available. A 
two-sample test of proportions was used for discrete 
level data to analyze across cohorts. Data were 
analyzed using SPSS Version 11.0 (SPSS, Inc., 
Chicago, IL). The a priori level of significance was 
set at alpha equal to 0.05. Qualitative data were 
grouped by theme. The University of Arizona 
Human Subjects Protection Program declared this 
project exempt. 

RESULTS 
Subjects 

The graduating class of 2006 consisted of 67 
individuals, 41 females (61%) and 26 males (39%). 
Upon admission, 38 (57%) had bachelors degrees 
and 1 (1.5%) individual had a masters degree. The 
average matriculant was 24.5 (SD=5.1 years) and 
ranged from 1 9 to 49 years of age. 

Number of hours spent weekly in various 
activities 

Students were asked to indicate a range for the 
number of hours spent in the following activities in a 
typical 7-day week: preparing for class or rotation 
(eg, studying, reading), working, participating in co- 
curricular activities (eg, professional organizations, 
social fraternities), and relaxing and socializing (eg, 
watching TV, partying, exercising). The hour ranges 
were: 0, 1-5, 6-10, 11-15, 16-20, 21-25, 26-30, and 
more than 30 hours. The distribution of the range of 
hours spent weekly by students in various activities 
by professional year are shown in Table 1 . Except 
for the number of hours spent preparing for class or 
rotation and the number of hours spent working, the 
mode number of hours spent in the remaining 
activities appeared to be fairly constant across 
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professional years. The mode hour range spent PY2. The mode hour range for working increased in 
preparing for class or rotation appeared to decrease PY2 and PY3 and then decreased again in PY4. 
in the PY3 and PY4 cohorts relative to PY1 and 



Table 1 . Range of hours spent weekly by students in various activities 




Percentage of Students 


Professional Year 1 


Zero 


1 to 5 


6 to 10 


11 to 15 


16 to 20 


21 to 25 


26 to 30 


more than 


hours 


hours 


hours 


hours 


hours 


hours 


hours 


30 hours 


preparing for class/rotations 


0 


1.6 


17.7 


19.4 


33.9 


16.1 


6.5 


4.8 


working 


32.3 


3.2 


21.0 


16.1 


19.4 


3.2 


3.2 


1.6 


co-curricular activities 


13.1 


70.5 


11.5 


3.3 


0 


0 


1.6 


0 


relaxing/socializing 


1.6 


34.4 


37.7 


16.4 


6.6 


1.6 


0 


1.6 


providing care for dependents 


41.9 


22.6 


22.6 


4.8 


1.6 


0 


3.2 


3.2 




Professional Year 2 


Zero 


1 to 5 


6 to 10 


11 to 15 


16 to 20 


21 to 25 


26 to 30 


more than 


hours 


hours 


hours 


hours 


hours 


hours 


hours 


30 hours 


preparing for class/rotations 


0 


1.7 


19.0 


17.2 


31.0 


17.2 


1.7 


12.1 


working 


10.5 


1.8 


19.3 


24.6 


28.1 


10.5 


3.5 


1.8 


co-curricular activities 


10.3 


56.9 


25.9 


6.9 


0 


0 


0 


0 


relaxing/socializing 


0 


31.0 


36.2 


22.4 


5.2 


5.2 


0 


0 


providing care for dependents 


43.1 


22.4 


13.8 


5.2 


3.4 


3.4 


0 


8.6 




Professional Year 3 


Zero 


1 to 5 


6 to 10 


11 to 15 


16 to 20 


21 to 25 


26 to 30 


more than 


hours 


hours 


hours 


hours 


hours 


hours 


hours 


30 hours 


preparing for class/rotations 


0 


6.0 


35.8 


19.4 


20.9 


7.5 


4.5 


6.0 


working 


6.0 


0 


16.4 


25.4 


26.9 


16.4 


6.0 


3.0 


co-curricular activities 


9.0 


73.1 


13.4 


1.5 


0 


3.0 


0 


0 


relaxing/socializing 


3.0 


20.9 


35.8 


17.9 


11.9 


6.0 


1.5 


3.0 


providing care for dependents 


45.5 


16.7 


19.7 


6.1 


4.5 


1.5 


1.5 


4.5 




Professional Year 4 


Zero 


1 to 5 


6 to 10 


11 to 15 


16 to 20 


21 to 25 


26 to 30 


more than 


hours 


hours 


hours 


hours 


hours 


hours 


hours 


30 hours 


preparing for class/rotations 


0 


24.2 


24.2 


18.2 


10.6 


9.1 


6.1 


7.6 


working 


23.1 


3.1 


27.7 


10.8 


16.9 


12.3 


4.6 


1.5 


co-curricular activities 


44.8 


37.3 


13.4 


3.0 


0 


0 


0 


1.5 


relaxing/socializing 


1.5 


18.2 


36.4 


22.7 


13.6 


4.5 


3.0 


0 


providing care for dependents 


46.3 


14.9 


14.9 


9.0 


4.5 


1.5 


3.0 


6.0 



Table 2: Student ratings of overall quality of various programmatic components* 




Professional Year (PY) 




PY1 


PY2 


PY3 


PY4 




Mean (SD) 


Mean (SD) 


Mean (SD) 


Mean (SD) 


pre-pharmacy advising 


4.00 (0.99) 






4.26 (0.81) 


admissions process 


3.87 (0.91) 






4.15 (0.79) 


orientation week 


3.33 (0.96) 








availability of financial aid information 


3.17 (1.15) 


3.16 (1.22) 


3.25 (1.16) 


3.74 (0.92) 


availability of financial aid 1 


2.97 (1.22) 


2.89 (1.19) 


3.33 (1.18) 


3.65 (1.12) 


Dean's office staff 


4.36 (0.68) 


4.26 (0.74) 


4.32 (0.73) 


4.32 (0.77) 


office of student affairs 5 


3.95 (0.80) 


3.78 (0.85) 


3.81 (0.94) 


4.74 (0.51) 


interaction with faculty members 


3.90 (096) 


3.95 (0.87) 


4.16 (0.86) 


4.23 (0.74) 


instruction 


3.73 (0.96) 


3.43 (0.94) 


3.88 (1.01) 




interaction with assigned faculty advisor 








2.56 (1.45) 


instruction in 3 years of didactic portion of 
curriculum 








4.05 (0.83) 


availability of professional electives in didactic 








3.41 (1.16) 


portion of curriculum 








instruction in experiential portion of curriculum 








4.21 (0.79) 


availability of electives in experiential portion 








4.08 (0.92) 


Career Day Fair 








3.93 (1.25) 


* Students were asked to "Please rate the overall quality of the following aspects of the pharmacy program" on a 5-point 
scale: 1 = poor, 2 = fair, 3 = adequate, 4 = very good, and 5 = excellent. 

f p = 0.010 where PY4 was greater than PY1 and PY2 using a Tukey post-hoc test 

* p = 0.0001 where PY4 was greater than PY1 and PY2 using a Tukey post-hoc test 

§ p <0.0001 where PY4 was greater than PY1 , PY2, and PY3 using a Tukey post-hoc test 



Quality of various programmatic components 

The mean student ratings of the overall quality of 
various programmatic components are shown in 
Table 2. Five programmatic components were 



tracked across all 4 professional years. The 
remaining 9 programmatic components were 
examined at the most logical points in the 
curriculum (eg, the Career Day Fair was only 
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offered during PY4 and thus only asked at the 
conclusion of PY4). The mean ratings of overall 
quality of the Dean's Office staff and interaction with 
faculty members remained constant across all 4 
years. The mean rating of the overall quality of the 
availability of (p=0.001) and information on 
(p=0.0001) financial aid was greater in PY4 than 
PY1 . Students rated the overall quality of the Office 
of Student Affairs highest in PY4 compared to PY1 , 
PY2, and PY3 (p<0.0001). The mean rating of 
instruction was lower in PY2 versus PY3 (p=0.035). 

In comparing programmatic components only 
examined for PY4, the 2 lowest rated programmatic 
components were the overall quality of the 
interaction with the assigned faculty advisor and the 
availability of professional electives in the didactic 
portion of the curriculum (p<0.0001). The overall 
interaction with faculty members was rated 
significantly higher than was the interaction with 
assigned faculty advisor (p<0.0001). The mean 
rating of elective availability in the final year was 



higher than the rating of didactic elective choices 
(p<0.0001). The lack of availability of professional 
electives in the didactic portion of the curriculum 
was the second most frequently mentioned aspect 
of the pharmacy program that they would want to 
improve. 

Rating quality of relationships among students, 
staff, and family 

The rating of the quality of relationships among 
students, staff, and faculty, is shown in Table 3. 
There was no difference across professional years 
for the student ratings of the quality of relationships 
with staff or with faculty. The mean rating of the 
quality of relationships with other students was 
higher for PY4 compared to PY1 (p=0.047). These 
ratings appear to be consistent with the qualitative 
comments provided by students about what they 
liked the most about their pharmacy education with 
faculty ranking the highest in number of mentions 
and friendships/classmates ranking fourth. 



Table 3: Students rating of quality of relationships among students, staff, and faculty* 




Professional 

Year 1 
Mean (SD) 


Professional 

Year 2 
Mean (SD) 


Professional 

Year 3 
Mean (SD) 


Professional 

Year 4 
Mean (SD) 


relationships with other students 1 T 


5.63 (1.07) 


5.67 (1.29) 


5.84 (1.26) 


6.16 (1.10) 


relationships with administrative personnel and 
offices 5 


5.79 (0.98) 


5.91 (0.94) 


5.87 (1.19) 


6.18(1.04) 


relationships with faculty members" 


5.59 (1.22) 


5.70 (1.07) 


5.97 (1.15) 


5.96 (1.07) 


* Students were asked to, "Thinking of your own experience over the past year, please rate the quality of these relationship 
on the following seven-point rating scales" 

f Anchored at 7 = "Friendly, supportive, sense of belonging" and 1 = "Competitive, uninvolved, sense of alienation" 

* Dunnett post-hoc test showed PY4 > PY1 (p = 0.047) 

§ Anchored at 7 = "Helpful, considerate, flexible" and 1 = "Rigid, impersonal, bound by regulations" 

" Anchored at 7 = "Approachable, helpful, understanding, encouraging" and 1 = "Remote, discouraging, unsympathetic" 



Table 4: Descriptive statistics on pharmacy intern employment status and practice setting information 




Professional 


Professional 


Professional 


Professional 




Yearl 


Year 2 


Year 3 


Year 4 




(n = 62 )* 


(n = 58 )* 


(n = 67 )* 


(n = 67 )* 


Currently working in a pharmacy setting 


35 (56.5%) 


49 (84.5%) 


59 (88.1%) 


44 (65.7%) 


Practice settings for those indicating that 
they were working in a pharmacy setting 










Community (chain/retail) 


23 


37 


51 


31 


Community (independent) 


2 


1 


0 


3 


Hospital/institutional 


9 


7 


5 


5 


More than 2 practice settings 


0 


3 


2 


0 


Other 


1 


1 


1 


5 


* sample size varied depending on attendance in class for day of administration of instrument 



Student employment status and practice setting 
information 

The descriptive statistics for the proportion of 
students working in a pharmacy practice setting for 
each professional year are shown in Table 4. A little 
more than half the class (56.5%) worked in a 
pharmacy setting during PY1. The proportion of 
students working increased in PY2 (p=0.030) and 
remained steady in PY3. In PY4 the proportion of 
students working decreased compared to PY2 and 
PY3 (p=0.0005). The greatest proportion of 
students worked in the chain/retail community 
pharmacy setting across all 4 professional years. 
Based on information collected by the Student 
Affairs Office for those that were able to indicate 



their plans after graduation approximately 50% 
entered chain/retail community pharmacy practice 
as their first job upon licensure. 

Additional Qualitative Feedback from Students 

In final data collection, students were asked to list 
the 3 things they liked the most about their 
pharmacy education. The most frequently 
mentioned aspects of their pharmacy education 
students liked the most listed in descending order 
were: the faculty, experiential rotations, the 
curriculum as a whole, friends/classmates, and 
student organizations/extracurricular activities. 
Students were also asked to list the 3 things that 
they liked the least about their pharmacy education. 
The most frequently mentioned aspects of their 
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pharmacy education students liked least listed in 
descending order were: the longitudinal drug 
information rotation, the physical facilities (eg, small 
classrooms, uncomfortable seating), student 
reflective portfolios, and the senior capstone project. 
The final open-ended item asked students to list the 
3 things they would do to improve their pharmacy 
education. The most frequently mentioned aspects 
students listed as areas they would improve in 
descending order were: rotations offered earlier in 
the curriculum, improved availability of electives, 
and the provision of formal preparation for the 
Board licensure examinations. 

DISCUSSION 

The change in the shape of the distribution and 
mode number of hours working for pay off campus 
decreasing in PY4 was expected due to the 
increased time demands and required attendance 
during the experiential portion of the curriculum. 
This was confirmed by a drop in the number of 
students indicating current employment status 
during PY4. Of those students that did work in a 
pharmacy setting during pharmacy school, the 
majority did so in a community setting. Given the 
large number of community pharmacies in the 
Tucson and Phoenix areas, this was not 
unexpected. By PY3, 89% of students indicated that 
they were working in a pharmacy setting which 
could be a potential double-edged sword which in 
the past has been a concern of the faculty. While 
having the vast majority of students working by PY3 
could provide further reinforcement of material and 
provide opportunities to complete outcomes based 
competency statements it has to be balanced with 
appropriate priority setting on the part of the student 
such as not working so many hours that it detracts 
from coursework or affects attendance. 

The apparent decrease in the mode number of 
hours spent preparing for class/rotations dropping in 
PY3 and PY4 compared to PY1 and PY2 was 
unexpected given the increased course load 
particularly in PY4 with a longitudinal drug 
information rotation running concurrently with other 
experiential rotations, a senior capstone project 
requires that a proposal process that begins during 
PY3, in addition to the annual requirement of the 
submission of a student reflective portfolio. 
Numerous responses from students to the open- 
ended items in listing the longitudinal drug 
information rotation, senior capstone project, and 
student reflective portfolio as the most disliked 
aspects of the pharmacy program indicated that 
they took too much time. The qualitative statements 
do not appear to agree with the overall ratings of 
how as a group they spent their time in PY4, 
spending apparently less in preparation for 
class/rotations than in PY1 and PY2. The item may 
not have reflected actual time spent for rotation 
preparation given that some of the time spent 
preparing may have been incorporated into the 
rotation time. The qualitative item on the instrument 
may have indicated a potential problem with the 
way subjects were being asked to quantify their 
preparation time. 



The higher rating of the Student Affairs Office in 
PY4 may be the result of the Student Affairs Office 
serving as one of the main points of contact while 
students are on rotation, helping to coordinate 
ensuring students have the required paperwork 
required for licensure. The Student Affairs Office 
also serves as the focal point for planning 
graduation festivities and the actual administration 
of the exit survey is conducted during "senior week" 
where all PY4 students are together one final time 
for a week prior to graduation. It is possible there 
was also somewhat of a halo type effect given the 
timing of the administration. The increased rating of 
the availability of and information on financial aid in 
PY4 relative to PY1 could have resulted from an 
increased number of named scholarships available 
to students in their final year, the increased dollar 
amount of awards as students progress further in 
the curriculum, changes instituted simplifying 
application procedures, and/or increased skill at 
navigating the financial aid system by the final year. 

The low rating of interaction with assigned faculty 
advisor is a continuing concern. In spring 2006, 
faculty advisors assumed the responsibility of 
grading their advisees' student portfolios. It is hoped 
this increased and sustained interaction will result in 
a higher rating of this program component. The 
finding that graduates felt that there were 
insufficient professional electives has also been 
encountered in previous years' exit surveys. Two 
new didactic electives were developed and offered 
in academic year 2005-2006. Faculty must have the 
time, desire, and incentives to provide elective 
offerings. 

Of the qualitative feedback from students, wanting 
earlier rotations, the provision of formal preparation 
for the Board licensure exams, and the need for 
improved physical facilities had not made the top 5 
in previous graduating classes. The physical 
facilities could be an artefact of the transition to the 
new UACOP building that occurred while the class 
of 2006 was on rotations which solved many of the 
concerns they listed such as cramped classrooms 
and certain seating areas in poor condition. The 
lack of formal introductory pharmacy practice 
experiences should be addressed with necessary 
curricular reform through more explicit requirements 
in Standards 2007 as to what constitutes 
introductory practice experiences. The listing of the 
provision of more formal preparation for the 
licensure examinations was unexpected and 
seemingly unfounded as outcomes were no 
different from previous years. 

CONCLUSION 

This longitudinal study provided valuable 
information on student perceptions of various 
programmatic components as well as in how 
students spend their time. College faculty and 
administration have been given summaries of the 
findings to be used in the next accreditation self- 
study deliberations and on-going curricular 
evaluation. The College's Curriculum and 
Evaluation and Special Studies Committees have 
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used relevant sections and previous exit survey CONFLICT OF INTEREST 
results in their on-going deliberations. None declared. 
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The views expressed in this paper are those of the 
authors and do not represent those of the American 
Association of Colleges of Pharmacy. 
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